Systems and Methods for Improved Cellular Quality of Experience

ABSTRACT

Systems and methods for providing an improved cellular user quality of experience (QoE) are disclosed. The system can comprise a database from multiple data points to monitor and analyze cellular user experiences holistically. The system supplements conventional quality of service (QoS) metrics with user-side, application provider, and internet provider data, among other things. The data can be used to create highly granular service maps. The data can also be used in methods for analyzing and solving network issues, including slowdowns, dropped calls, and network availability are also disclosed. Improved analysis of network, user equipment (UE), and application issues can locate and solve QoE issues, improving cellular customer satisfaction, retention, and loyalty.

BACKGROUND

The combination of cellular technology and modern smart phones, amongother things, has created an explosion in the volume of data that isprovided wirelessly to users. Various applications including, forexample, Facebook®, You Tube®, voice calling, and texting all requiredigital bandwidth. This data can be provided over various cellularnetworks (e.g., 2G, 3G, 4G, and 4G LTE) and can also be provided, oraugmented, by various wireless networks (e.g., 802.11b/g/n).

Conventionally, the quality of a user's cellular experience has beenmeasured using quality of service (QoS) metrics. These tend to beobjective, technical, network measurements and can include, for exampleand not limitation, signal strength, error rates, bandwidth, throughput,transmission delay, availability, and jitter. And while these metricscan accurately measure the objective performance of a network, networkperformance does not necessarily translate into quality of experience(QoE) for the user.

If, for example, a user attends a football game at a stadium withmultiple cellular access points, QoS metrics may indicate that the userhas excellent signal, and that the system has enormous throughput (dueto multiple access points). If a large number of people in the stadiumare using their phones simultaneously, however, the user may nonethelessexperience dropped calls, slow download speeds, and/or sluggish responsefrom various applications. Thus, many users may have a poor QoE despitethe fact that QoS metrics indicate that the system is operatingnominally. As a result, QoS metrics do not and cannot address someimportant issues.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is set forth with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different figures indicates similaror identical items or features.

FIG. 1 is a schematic of a conventional cellular/wireless network.

FIG. 2 is a schematic of a conventional cellular coverage map.

FIG. 3A is a schematic of a cellular network performance map, inaccordance with some examples of the present disclosure.

FIG. 3B is a graphical user interface for providing a user's cellularperformance and service recommendations, in accordance with someexamples of the present disclosure.

FIG. 3C is another graphical user interface for providing a user'scellular performance and service recommendations, in accordance withsome examples of the present disclosure.

FIG. 4A is a graphical user interface for providing a user's equipmentand network performance, in accordance with some examples of the presentdisclosure.

FIG. 4B is a graphical user interface for providing a user's applicationperformance, in accordance with some examples of the present disclosure.

FIG. 5 is a system diagram for a system for gathering and compiling datarelated to overall cellular quality of experience (QoE), in accordancewith some examples of the present disclosure.

FIG. 6 is a system diagram for a user equipment (UE) performanceapplication, in accordance with some examples of the present disclosure.

FIGS. 7A and 7B are a flowchart depicting a method for locating andreporting network, user equipment (UE), and application faults, inaccordance with some examples of the present disclosure.

DETAILED DESCRIPTION

Examples of the present disclosure relate generally to cellular userquality of experience (QoE), and specifically to systems and methods formeasuring, tracking, and improving user's QoE using multiple metrics. Insome examples, the system can comprise an application on the user'sequipment (UE) to measure user side network response times, andprocessor and memory usage, for example, to enable the system topinpoint causes of poor QoE. In some examples, the system can combineelements of QoE data with quality of service (QoS) data to identifyproblems and offer solutions. The system can also include one or moreuser interfaces that provide indications of current QoE conditions tousers.

To simplify and clarify explanation, the disclosure is described hereinas a system and method for analyzing, tracking, and fixing issuesrelated to a user's cellular QoE. One skilled in the art will recognize,however, that the disclosure is not so limited. The system can also beused, for example and not limitation, with other types of networks,including other wireless and wired networks. In addition, the disclosureprovided below is related to current cellular technologies (e.g., 2G,3G, 4G, and 4G LTE (Long Term Evolution)), but is equally applicable toother network technologies, including technologies developed after thisdisclosure.

The metrics, methods, and steps described hereinafter as making up thevarious elements of the present disclosure are intended to beillustrative and not restrictive. Many suitable datasets, applications,and networks that would perform the same or a similar function as thesystems described herein are intended to be embraced within the scope ofthe disclosure. Such other systems and methods not described herein caninclude, but are not limited to, systems, networks, and technologiesthat are developed after the time of the development of the disclosure.

QoS, generally speaking, is the monitoring of the specificinfrastructure components such as, for example and not limitation,servers, routers, or network traffic (e.g., IP packets, transportstream, etc.). QoS metrics are generally device or transport-orientedand thus measure objective factors including, for example, CPU, memoryuse, packet loss, delay, or jitter.

Unfortunately, QoS cannot evaluate the quality of the network as seenand perceived by the end user. Because current infrastructures aredesigned to be robust and redundant—with back-up services, alternativenetwork routing, and error correction—problems on the network or devicelayer do not necessarily cause a service-layer issue for the end user.Alerts triggered because a router is down might not signify a serviceoutage for the user, especially if traffic is correctly rerouted, butmay result in delays for the user.

QoE performance indicators, on the other hand, are user-centric. Theyinclude metrics such as, for example and not limitation, time todownload a webpage, access an application, place a phone call, change aTV channel, log into an interactive service, and measuring video & audioquality. This helps take the guesswork out of impact assessment andprovides factual information to, for example, drive operations,prioritize investments (e.g., in infrastructure), and for UE selection.

One way to evaluate the quality delivered to a user is for the providerto connect to the service like a user. Unfortunately, due to the numberof different types of users, UE, locations, and infrastructuresinvolved, it is difficult to collect enough data to make this ameaningful exercise. Another solution, using a multipronged approach,can include an application on the UE to monitor user-side experiencecombined with data from, for example, internet service providers (ISPs),content providers (e.g., Facebook, YouTube, etc.), and even direct inputfrom users.

Of course, both QoE, for subscriber-level monitoring, and QoS, forequipment troubleshooting and root-cause analysis, can be usedcooperatively to maximize customer experience.

As mentioned above, conventional network analysis tends to rely onpurely technical QoS metrics. These systems use metrics such as signalstrength, error rates, bandwidth, throughput, transmission delay,availability, and jitter. The cellular and wireless networks in usetoday, however, have many components outside the cellular networkitself. In addition, QoS metrics are often unable to identify certainissues because they are not capable of measuring overall systemperformance. In other words, QoS metrics are unable to measure theoverall performance of the system, from the user's perspective,precisely because they do not monitor many components outside thecellular network.

A user may be frustrated with the speed of service in a large stadiumdue to the sheer number of users on the same network. With respect toQoS metrics, however, the system may appear to be operating nominally¹,if at higher than normal usage rates. In addition, if a user experiencesa dropped call, for example, but hangs up their handset within apredetermined time period (e.g., 30 seconds), a QoS system maynonetheless register a successful call because no UE-side data isavailable. There are many such instances where the QoS system indicatesgood QoS, yet the user experiences poor QoE. ¹ Nominal is used here inthe engineering sense. In other words, a system is operating nominallyif, for example, it is operating at a predetermined download speed plusor minus some tolerance (e.g. 20 Mbps±2 Mbps). Because different typesof networks have widely different data and error rates, this term is notintended to be limiting, but merely to convey a range of normaloperation.

From a client retention and satisfaction standpoint, it is QoE that ismore important than QoS. In other words, a user with a dropped call doesnot care that the network appears, from a QoS standpoint, to beoperating nominally. All the user knows is that their call dropped.Dropped calls are a major cause for users switching from one cellularcarrier to another, for example. Similarly, if a user has “five bars”showing on his cell phone, but his phone takes ten minutes to downloadan e-mail, he will be similarly dissatisfied.

To this end, examples of the present disclosure relate to systems andmethods for combining QoS and QoE metrics, among other things, to locateand eliminate system bottlenecks and/or offer alternativeconfigurations. The system can comprise one or more applications on theUE to provide user-side data. The system can also collect data fromcellular network providers (e.g., T-Mobile) and network contentproviders (e.g., Facebook and YouTube), and survey data from the user.In some examples, the system can provide one or more graphical userinterfaces (GUIs) to provide users with real-time performance data forone or more networks and/or applications. In other examples, the systemcan offer highly granular performance maps to enable consumers to solvenetwork issues or make equipment and provider choices. Examples of thepresent disclosure can also comprise a method directed to the analysisof multiple data sources to locate, and suggest solutions for, networkbottlenecks.

FIG. 1 depicts a conventional cellular network including 2G 105, 3G 110,and 4G LTE 115 components. As is known in the art, data can be routedfrom the internet or other sources using a circuit switched modemconnection (or non-3GPP connection) 120, which provides relatively lowdata rates, or via IP based packet switched 125 connections, whichresults is higher bandwidth. The LTE system 115, which is purely IPbased, essentially “flattens” the architecture, with data going straightfrom the internet to the Service Architecture Evolution Gateway (SAE GW)130 to evolved Node B transceivers 115, enabling higher throughput. ManyUEs 135 also have wireless local area network (WLAN) 140 capabilities,in some cases enabling even higher throughput.

QoS systems 150 generally only cover the cellular system side components(e.g., gateways 130, towers 115, etc.). Notably, one major componentthat is outside the purview of the QoS system 150 is the UE 135. Thiscan include a wide variety of phones, smart phones, tablets, andcomputers and can also include a wide variety of operating systems(e.g., Windows, iOS, Android, etc.). UE also includes a wide variety ofphysical layouts, which can affect, for example, antenna reception,processor speed, and battery life. Thus, because conventional QoStechniques are not capable of monitoring errors and issues arising fromUE 135, a major diagnostic component is missing from the system.

In addition, application providers 155 such as, for example and notlimitation, Twitter, Facebook, Instagram, and You Tube provide mobileapplications for use on UE 135. Some, like You Tube, require relativelyhigh data bandwidth to stream or download videos, for example, but areless dependent on response times. Others, like online games, may insteadrequire fast response times and fast cellular handovers (i.e., handingthe UE 135 from one cellular tower 115 to the next as the user travels)to maintain continuity of play. Still other applications, such as makingvoice calls, have much lower bandwidth demands, but may require very lowerror rates to prevent dropped calls and audio artifacts, for example.

Each of these applications can be easily handled by a strong 4G LTEconnection. Unfortunately, the equipment required to provide 4G LTEcoverage has not yet been fully deployed and thus, a 4G LTE network isnot always available. This also does not address the stadium issue,mentioned above, because there are limits to the number of users anynetwork can handle. In addition, in certain areas, geological,electrical, and meteorological conditions may block or interfere withsignals. Thus, in some cases, it may be beneficial to use legacy 2G or3G connections, which may be underutilized because, for example, theyare thought to be outdated. In some cases, it may also be useful tosupplement or replace cellular bandwidth with WLAN 140 bandwidth. Asdiscussed below, examples of the present disclosure can enable cellularservice providers to classify users, identify needs, locate bottlenecks,and offer solutions to network issues.

As mentioned above, a problem with a conventional QoS system 150 is thatit does not take into account all, or even most, of the variables thatcan cause issues. The QoS system 150 does not monitor UE 135 sideissues, for example. Thus, if a particular phone or application iscausing issues, the QoS system 150 is unable to detect these issues.Similarly, the QoS system 150 does not receive data from applicationproviders 155. Thus, the QoS system 150 may be capable of providing highspeed and bandwidth, but Application A is nonetheless dragging on the UE135. Unfortunately for cellular service providers, the user does notknow (or care) where the problem lies, they just want things to work.

As a result, different applications may be more or less suitable fordifferent connection types and vice-versa. To aid in decision making andproblem solving, therefore, examples of the present disclosure cancomprise a highly granular, or even personalized, service map. As shownin FIG. 2, conventional coverage maps are useful, but tend to only showgeneral swathes of information. The map may not provide information withhigh enough granularity to determine that although you live in an areawith generally good coverage, for example, you and your neighbor's houseare located behind a hill that almost entirely blocks the signal.Similarly, the wiring and construction of some houses can block asignificant portion of cellular signals.

To this end, data collected from individual users, cellular providers,and content providers can be aggregated into a database that can be usedto create a highly granular map. As shown in FIG. 3A, for example, themap can include multiple network types 310 and, in some examples, can begranular down the individual location 305 level. The map 300 can use acombination of QoS and QoE metrics (the “combined data”) to provide apredicted QoE for users in that location. In other words, QoS metricsmay indicate that a user would have full 4G LTE access at a particularlocation. QoE metrics from previous or contiguous users, however, mayindicate geographical issues or other anomalies that prevent acceptable4G LTE coverage.

As a result, in some examples, the map 300 can show only network typesthat are predicted to be available in a particular location or at aparticular location 305 based on the combined data. Locations cancomprise many particular locations where data regarding cellular serviceis helpful such as, for example and not limitation, houses, buildings,businesses, and a variety of public use areas (e.g., parks and airports,for example, with public access). In other examples, the map 300 canprovide network performance predictors 315, such as grades (e.g., A, B,C, D, and F), red light/green light, check marks, or other networkperformance predictors 315 for the various networks available at aparticular location.

4G LTE coverage at a particular location 305 a may provide a poor QoE,for example, but 2G and 3G QoE is excellent. This can be caused by thelocation 305 a itself, geographical conditions, because the 3G networkuses a different cell tower, or other technical reasons. Theseanomalies, however, are not detected from the network, or QoS, side, butcan be readily detected using QoE metrics. If the user in question makespredominantly voice calls, therefore, they may be well served by the 3Gnetwork. If the user is a heavy data user, on the other hand, they maydecide to use a WLAN connection at home, which sidesteps the 4G LTEcoverage issue.

Conversely, the map can enable the service provider (e.g., T-Mobile) todetermine if network modifications are required or if the problem is outof their control (e.g., geographical or meteorological). In other words,due to the high granularity of the map 300 and the added input from theUE and/or the user, the service provider can determine whether acoverage issue is widespread, localized, or even a particular location305. If the issue is widespread despite the presence of a sufficientnumber of cell towers, for example, the service provider may decide toinstall additional cell towers. If, on the other hand, the issue islocalized or individual, the service provider may decide to providewireless routers, for example, to affected customers.

In some examples, the map 300 can also include network issue predictors320. If, based on user experience with download times, throughput times,or other factors, a particular network type does not generally operatenominally in a particular location (i.e., response time is slow,throughput is low, etc.), for example, the map 300 can display thecause, if known. The map 300 can have a geographical issue indicator 320a indicating that a geographical problem exists. This can be due to thelocal topography, large buildings, or other physical features that blockcellular signals. Similarly, if the issue is caused by distance, thoughthe QoS metrics would indicate otherwise, for example, the map 300 candisplay a distance issue indicator 320 b. This may be due to localinterference sources, for example, that reduce the effective range ofthe cellular system in that area despite the fact that a location iswithin the normal operating range for that cellular system.

Because of the granularity of the map 300, the map 300 can also have alocation issue indicator 320 c. In other words, if a particular location305 should have good reception, based on the locations 305 around it,signal strength at the location 305, and/or other factors, but does not,the location 305 itself may be causing the issue. This could be due towiring, construction, or other factors in the location, for example,blocking cell signals (e.g., the wiring in the house may effectivelycreate a Faraday cage). In this case, the map 300 can display thelocation issue indicator 320 c prompting the user to investigatefurther.

In other embodiments, the map 300 may be customized for the type of UE135. If QoE issues have been reported in the location 305, for example,but only for a particular UE (e.g., Device A); the map 300 can betailored to the user with this type of UE 135. In other words, the map300 can display one set of predictors 315, 320 for Device A and anotherset for Device B. In this manner, a user is not disappointed by poor QoEdue to an underlying UE 135 issue.

In other examples, as shown in FIGS. 3B and 3C, the system can comprisea GUI 350 to enable the user to enter their address and receive feedbackfor coverage at their house. So, for example, the user can enter theiraddress in an address field 355 and receive feedback 360 based on thecombined data as to which networks are available and will operate in asatisfactory manner. In some examples, the feedback 360 can comprise asimple check mark or x-mark, as shown. In other examples, the feedback360 can comprise, for example and not limitation, a signal meter, a redlight/yellow light/green light indicator, or a grade.

In some examples, the GUI 350 can also comprise one or more inputsregarding the user's cellular habits. In some examples, as shown in FIG.3B, the inputs can comprise a list of applications 370 that the user isasked to rank. The applications 370 can include, for example, e-mail andinternet browser applications, social media sites, and video channels.In other examples, the user can simply be asked to rank the type of data375 they use most, as shown in FIG. 3C. The types of data 375 cancomprise, for example and not limitation, voice calls, e-mail, internet,and video. In still other examples, the user can simply indicate whatapplications or types of data they use with a check mark or otherindicator.

If applicable, the GUI 350 can also comprise a special servicerecommendation 365. If, as shown in FIG. 3B, based on the combined datathe user is predicted to have strong 4G LTE coverage at his location305, for example, no special service recommendation 365 may be requiredbecause the user should have excellent functionality. If, on the otherhand, as shown in FIG. 3C, the user is a heavy data user and ispredicted to have poor data network coverage, the special servicerecommendation 365 can include a WLAN connection, wireless hotspot, orother solution to provide the desired network performance.

Examples of the present disclosure can also comprise a GUI 400 toprovide real time, or almost real time, statuses of one or moreapplications 405 or network interfaces 410 on the UE 415 based on thecombined data. As shown in FIG. 4A, for example, in some examples, theGUI 400 can essentially comprise a “dashboard” on the UE for variousnetwork connections, for example. The GUI 400 can provide a speedometer420, fuel gauge 425, or other similar representation to depictpredictions for the current status of network speeds, response times,available bandwidth, error rates, etc. These metrics can be based on,for example, actual measured performance combined with UE data and/orfeedback from users proximate the user. In some examples, the GUI 400can provide these metrics for, for example and not limitation, available2G, 3G, 4G LTE, and WLAN networks.

The GUI 400 can also include fuel gauges 425, or other indicators, foron-board UE 415 metrics such as, for example and not limitation,processor usage 425 a, available buffer 425 b, and available memory 425c. Thus, if a user experiences slow response when using Application A,for example, the GUI 400 may indicate that there is little or noremaining buffer 425 b causing Application A to stop and rebufferexcessively. The user may wish to restart his phone, stop otherapplications, or create a larger page file, if possible. Similarly, ifthe GUI 400 indicates excessive processor utilization 425 a, the usermay again wish to stop unused applications or restart. If the user'sprocessor is persistently overworked (e.g., he is a gamer or simply hasan older phone), he may wish to upgrade his UE 415 to one with a morepowerful processor.

In other examples, as shown in FIG. 4B, the system 450 can provide amore high level view of system performance. In other words, some usersmay be uninterested in why a particular application is running slowly,but simply want to know that it is running slowly. The user may not carewhen they see a particular video, for example, or need immediate statusupdates from a social networking site. In the stadium example, forexample, rather than opening a social networking site only to befrustrated by the slow response and upload times, the user can simplyglance at their home screen 452. On the home screen 452, standardapplication icons 465 can be supplemented with indicia of how thatapplication is predicted to perform currently, whether due to network,application, or UE issues. The application performance predictors 455can comprise, for example and not limitation, colored indicators 455 a(e.g., red, yellow, and green lights), check marks and x's 455 b, orgauges (e.g., an application speedometer overlay 455 c). As above, theapplication performance predictors 455 can combine QoS metrics relatedto measured network performance with, for example, data from UEapplications and user input.

In some embodiments, the system 400 can utilize real-time user inputregarding current QoE for various applications and networks. So, forexample, if a plurality of users are in the aforementioned stadium andone or more of them reports slow performance for Application A, forexample, this data can be integrated into the application performancepredictors 455. Similarly, issues reported by application providers(e.g., server outages) can also be included to provide accuratepredictions of a user's likely QoE.

In some examples, the application performance predictors 455 can alsoinclude an application issue indicator 460. In other words, in additionto providing information about the performance of a particularapplication, the application issue indicator 460 can also provideinformation about the current bottleneck by providing letters orsymbols, for example, indicative of various system components. If, forexample, Application A is slow because of issues with the Applicationitself (e.g., the network content provider reports a denial of serviceattack to the system), the issue indicator can display a “P” 460 a for“provider.” If, on the other hand, the issue is with the 4G LTE network(e.g., the cellular provider reports tower issues), for example, theissue indicator can display a “4G” or “N” 460 b for network. This canenable the user to try other available networks (e.g., 3G), ifavailable, or simply wait for the issue to clear. Similarly, if the UE415 is the issue, the issue indicator can display “UE Issue,” 460 c, orsimilar. This may prompt the user to check the GUI 400 dashboard to seeif the problem can be easily rectified (e.g., with a restart) on the UE415. Of course, the issue may be caused by the application itself. Inthis case, the issue indicator can comprise an “A” 460 d, for example,for application, or similar.

In still other examples, the system 450 may be unable to identify theissue, or there may be multiple contributing factors. In this scenario,the system 450 can display a question mark 460 b, for example. The usercan then wait to see if the issue(s) resolves itself, check the GUI 400dashboard, restart the UE, or call technical support, if desired. Insome examples, applications that are predicted to be working nominallycan be indicated with a check mark 455 b, or can simply have theapplication icon 465—i.e., with no indicia (indicating that there are noproblems).

Examples of the present disclosure can also comprise a system 500 formonitoring and analyzing the entire QoE domain, including QoS metrics.In some examples, the system 500 can comprise a processor 505 forreceiving a plurality of inputs and generating a plurality of outputs.In some examples, the processor 505 can comprise a computer processor,laptop, tablet, application specific integrated circuit (ASIC), or fieldprogrammable gate array (FPGA).

The processor 505 can receive data from a number of sources to provideimproved overall system analysis. The processor 505 can receive data,for example, from application providers 515, which can includeapplications like, for example and not limitation, Twitter, Facebook,Instagram, and YouTube. The application providers 515 can provideinformation about how users access their applications, locations of use,and other metrics. This can enable the system 500 to identify issuescaused by the applications or application providers themselves, asopposed to network, or other, issues.

Similarly, in some examples, the processor 505 can receive data from anapplication 520 on the UE, as discussed below. The application 520 cancompile data from the UE itself including, but not limited to, signalstrength, location, processor, buffer, and memory usage, talk time,internet connection time, download speeds, and data usage. This canenable the system 500 to identify issues caused by the UE, as opposed toother factors. If the user is a gamer, for example, and the UEapplication 520 consistently reports high processor usage and/or highlatency from the UE application 520, the system 500 can determine thatthe UE is a possible bottleneck, which may be addressed with a newphone, for example.

The processor 505 can also receive data from one or more internetservice providers (ISPs) 525. The ISP data 525 can provide informationabout UE internet data usage (e.g., on WLAN connections), internetprovider network traffic and speeds, and overall internet speeds (e.g.,the internet itself may actually slow down due to heavy usage). ISP data525 can enable the system 500 to identify ISP bottlenecks, averagespeeds for ISPs, reliability, etc. This can enable the system 500 toidentify ISPs with and without network issues. This can lead topartnerships with those ISPs that provide the best service, for example.

The processor 505 can also receive data from one or more cellularproviders 530. Cellular providers 530, such as T-Mobile, can provideinformation about cellular system traffic and loads, network downtime,maintenance, and user profiles. User profiles can include, for exampleand not limitation, UE, location usage, and cell tower network usage(e.g., 3G vs. 4G LTE). This can enable the system 500 to quicklyidentify problems with cellular providers.

In addition, the system 500 can also utilize conventional network (QoS)metrics 535. The QoS metrics 535 can be used to identify network-sidefactors such as, for example, network bandwidth, response times,handover times, and usage. If the QoS metrics 535 indicate that aportion of the network has consistently high usage rates, for example,additional bandwidth capacity may be needed for that portion of thenetwork. This may be because that portion of the network is located in aparticularly populous area (e.g., Manhattan) or because that portion isin an area of higher than normal usage (e.g., Silicon Valley).

The processor 505 can also receive data directly from users. In someexamples, for example, user's e-mails, phone calls, and other contactswith service providers 570 may be included. So, for example, if a usersends an e-mail or contacts a help desk, data related to the user'sissue can be included. In some examples, the system 500 can also includean online application 575, an application on the UE 580, a survey, orother means for gathering data from users. The application 575 caninclude questions regarding a user's perceived QoE (i.e., regardless ofthe raw data). Thus, if a user is not satisfied with their service, atechnical problem can be identified based on the QoS data. If notechnical problem can be identified, the system 500 can initiate a callto the user, for example, to identify possible user or setting issues.If a user is not satisfied with an application simply because he cannotfigure out how to use it, for example, these issues can be addressedbefore the issue escalates and/or the user switches carriers.

In some examples, the processor 505 can compile the combined data (QoEand QoS data) into a database 510. In some examples, the database 510can contain data for both normal system operations (i.e., when thenumber of dropped calls, errors, delays, etc. is nominal) and data forsystem faults. In some examples, the processor 505 can identify andlabel database entries (i.e., either nominal data points or fault datapoints). The database 510 can then be used, for example, to generate theaforementioned performance map 300, among other things.

Because of the level of detail provided by the system 500, the map 300can contain multiple layers of data regarding predicted networkperformance, ISP performance, and UE performance, among other things.Thus, the map 300 can be used at multiple levels. The map 300 can beused by cellular users, for example, to determine what types of networksare available at home or at work to predict performance, identifyissues, and provide solutions, as applicable. Cellular providers can usethe map 300 to identify current, and predict future, areas of poorreception, areas of high usage, and/or to provide alternatives tocustomers. Application providers and ISPs can use the map to identifyusers for technical and marketing purposes, among other things.

The database 510 can also be used to identify and/or predict issues thatwould not otherwise be recognizable. Because the database 510 includedata from both the QoS and QoE sides of the cellular system includingthe cellular provider, network, applications, and the user; issuescaused by particular UE, applications, or cell towers, for example, canbe quickly identified. In addition, the database 510 can be mined toidentify potential future issues and provide location specific QoEpredictors. If, for example, a particular user has an increase in datausage, for example, this may expose a weakness in the user's connectionthat was not previously apparent. The database 510 can predict the issueand, in some examples, offer an alternative, fixing the issueproactively (or at least more quickly). These types of proactivemeasures taken by service providers create strong customer loyalty.

Using only QoS data, for example, could indicate that a particular celltower is dropping a disproportionate number of calls. This could leadthe provider to conclude that the tower has, for example, a defectiveantenna or other issue. If a particular type of UE (e.g., Device A)drops all (or most) of the calls on that particular cell tower, however,the system 500 can identify the problem as the UE and not the tower fromthe UE application 520, cellular provider 530, or combinations thereofThis data can be provided to the cell phone manufacturer, for example,to prompt a software fix, if applicable. Without QoE data forcomparison, conventional QoS metrics would not detect, let alonepinpoint, this problem.

The conglomeration of data from the system 500 in the system database510 can enable further analysis and predict future performance. So, forexample, if all, or almost all, of Device A drop calls in a particulararea, but other models of phones do not, the problem can likely benarrowed to the type of phone. This can enable analysis of software orsystem settings, for example, in an attempt to locate a solution. Ifnecessary, a software update can be implemented to fix the issue. Insome cases, the system 500 may recommend a different UE to avoid theseissues.

In some examples, the problem may occur in 4G mode, for example, but notin 3G mode. If a user may make predominately voice calls, therefore,switching to the 3G network in that cell (if available) may be a viablealternative. Of course, it is possible that local conditions are suchthat the problem cannot be easily resolved. For a customer that lives orworks in a cell that is overloaded or has difficult geography (e.g.,mountainous terrain), therefore, the solution may be to offer the user aWLAN router to enable the user to access VOIP and data via a WLANconnection. Alternatively, if the problem is unique to the phone, thesolution may be to offer the user a different model phone (at little orno cost, if necessary).

Examples of the present disclosure can also comprise an application 600for tracking one or more onboard UE functions. In some examples, forexample, the application 600 can run on the processor 605 of the UE andreceive various inputs from the UE. The application 600 can receiveinputs from the UE memory 610 and/or memory buffer 615, for example.This can enable the application 600 to monitor, for example, instant andaverage buffer 615 usage, actual download speeds (as opposed to thoseprovided on the QoS side), application response times, etc. If the userwatches a lot of internet video content, for example, a larger page fileor UE with greater memory capacity may be warranted. Similarly, in someexamples, the application 600 can receive inputs from the processor 605of the UE. This can enable the application 600 to monitor, for example,instant and average processor 605 usage, response times, processingtimes, etc. If the user is a gamer, for example, a faster processor maybe warranted to limit freezing and/or produce smoother graphics.

In some examples, the application 600 can also log internal errors anddropped calls 625. As mentioned above, QoS monitoring may be unable todetect a dropped call during certain conditions (e.g., the userimmediately hangs up after hearing the call drop). In contrast,monitoring dropped calls and error rates 625 from the UE-side can enablethe application 600 to log dropped calls that may otherwise gounnoticed. Comparing QoS data with UE side data 625, for example, canenable providers to pinpoint potential problems—i.e., to determine if itis a problem with the network or the UE. The dropped calls 625 could bethe result because of UE issues (e.g., manufacturing or softwareissues), for example, or a faulty cell tower, which may not be easilydetectable using other means.

In some examples, the application 600 can also receive inputs from theUEs transceiver or antenna system 620. This can enable the application600 to log information regarding signal strength, error rates, pingtimes, and other data. This data can be used for the performance map300, to identify faulty cell towers, and to identify faulty antennas andtransceivers 620. As before, if a particular type of UE shows atypicalreception in a certain area, the antenna design of the UE (among otherthings) may be the culprit. Of course, if all UEs on a certain towershow lower than expected signal, then the tower may have a ground fault,power issues, or failing transmitter, for example.

Examples of the present disclosure can also comprise a method 700 foranalyzing and locating network bottlenecks. As mentioned above,traditional QoS systems do not encompass the entire cellular/wirelesssystem. As a result, QoS systems may be unable to correct some systemicproblems because they do not see them. In other words, if the problem isthat a user requires a UE with a faster processor because of his usagepatterns, for example, a QoS system, which only looks at networkperformance, is unable to identify this problem. What is needed,therefore, is a method 700 that combines QoS network data with QoE datafrom UEs, internet content providers, and other sources to locate andresolve network bottlenecks.

In some examples, therefore, the method 700 can comprise retrievingfaults from the database 510 for further analysis, as shown at 702.Faults can be, for example and not limitation, drop calls, dropped datapackets, delays, and rebuffering. Because there are multiple types offaults, therefore, the method 700 can continue by identifying the typeof fault, as shown at 704. If the fault is a dropped call 704, forexample, the method 700 can determine if the cell tower associated withthe fault has a higher than normal dropped call rate, a weak signal, orother problem, as shown at 706. This can enable the system to identifyfaulty towers and to generate a tower report, as shown at 712. The towerreport 712 can enable the cellular provider to address maintenanceissues with the tower to return it to nominal operation.

If, on the other hand, the tower has a normal error rate, the method 700can determine if the particular UE has a higher than normal dropped callrate, a weak signal, or other problem, as shown at 708. If the UE has ahigher than normal drop rate, for example, the system can create a UEreport, as shown at 714. The UE report 714 can be provided to cellularproviders and/or cellular manufacturers. The UE report 714 can also becross-referenced against data from the application 600 to identify apossible faulty component (e.g., a bad batch of transceivers). This canenable the system to provide suggestions as to a possible resolution tothe problem. If the problem is a hardware problem, as with a faultytransceiver, the solution may be to replace the UE. If the problem doesnot appear to be hardware related, additional analysis may be needed tolocate software, or other, problems.

If the problem does not appear to be UE or tower related, the method 700can add the fault to an exceptions report for further analysis, as shownat 710. If, for example, a particular user's UE—as opposed to aparticular type of UE—shows up repeatedly in the exceptions report 710,for example, the fault may be the UE itself

Similarly, the method 700 may also determine that the fault is anapplication related issue (e.g., freezing, failing to open, or havingaccess issues), as shown at 716. If so, the method 700 can identifywhether any network issues existed at the time of the fault, as shown at718. Network issues 718 could arise from cellular network issues, suchas overloading, which can generally be derived from conventional QoSmetrics 535. Network issues 718 can also arise from internet issues,which can be identified from ISP data 525 discussed above. If thenetwork is determined to be the problem, the fault can be added to anetwork report 728, which may be divided between cellular and internetissues, for example.

If the fault is caused by a football game, as in the stadium scenario,in some examples, the cellular provider may decide to install moreaccess points at the stadium, for example. In other examples, thecellular provider may do nothing because it is not a prudent capitalinvestment because of the sporadic use. If a particular portion of thenetwork, either cellular or internet, repeatedly shows up on the networkreport 728, however, it may be desirable to increase the number ofaccess points, upgrade access points, or install larger internetconnections (e.g., larger trunk lines in that location).

If, on the other hand, the network was operating nominally at the timeof the fault, the method 700 can determine if the particular UE has ahigher than normal fault rate overall, as shown at 720. If the UE has ahigher than normal fault rate, for example, the system can create a UEreport, as shown at 714. The UE report 714 can be provided to cellularproviders and/or cellular manufacturers. The UE report 714 can also becross-referenced against data from the application 600 to identify apossible faulty or outdated component.

This can enable the provider to offer suggestions as to a possiblesolution to the problem. If the application is freezing because ofinadequate processor power or memory, this issue can be located byreferring to the application 600. If the processor was overloaded at thetime of the fault, for example, the solution may be to replace the UEwith a product with a more powerful processor. If the problem does notappear to be hardware related, but is common to a particular type of UE,additional analysis may be needed to locate software conflicts, orother, problems.

If the fault is not common to a particular type of UE, the method 700can determined if the problem is the application itself, as shown at722. The application 722 may simply have buggy code that does not runwell regardless of the UE. In some instances, the application 722 mayrepresent a security vulnerability, and the faults may be caused by avirus or other malware. In this case, providers may decide to report theapplication to, for example, the application publisher, advise users notto use the application, or remove the application from UEs and onlineapplication stores. The method 700 can add applications that repeatedlycause faults to an application report, as shown at 724, which can besupplied to cellular providers, content providers, and the publisher,among other things.

If the problem does not appear to be network, UE, or applicationrelated, the method 700 can add the fault to an exceptions report forfurther analysis, as shown at 710. The method 700 can enable systemfaults to be located even when they are outside conventional QoSsystems. The method 700 can enable the location, often in a proactivemanner, of problems, which can also lead to the solution. The method 700can enable providers to provide customers with a better QoE, increasingcustomer loyalty and retention and reducing costs associated withcustomer service and technical support, among other things.

While several possible examples are disclosed above, examples of thepresent disclosure are not so limited. For instance, while a systems andmethods for use with cellular phone systems has been disclosed, othersystems or subsystems could be analyzed in a similar manner withoutdeparting from the spirit of the disclosure. In addition, the locationand configuration used for various features of examples of the presentdisclosure such as, for example, the order of steps, the types ofcellular technologies, and the components monitored can be variedaccording to a particular network or application that requires a slightvariation due to, for example, the size or construction of the network,the communication protocols, or the UEs used. Such changes are intendedto be embraced within the scope of this disclosure.

The specific configurations, choice of materials, and the size and shapeof various elements can be varied according to particular designspecifications or constraints requiring a device, system, or methodconstructed according to the principles of this disclosure. Such changesare intended to be embraced within the scope of this disclosure. Thepresently disclosed examples, therefore, are considered in all respectsto be illustrative and not restrictive. The scope of the disclosure isindicated by the appended claims, rather than the foregoing description,and all changes that come within the meaning and range of equivalentsthereof are intended to be embraced therein.

What is claimed is:
 1. A computing device comprising: a display; one ormore processors; one or more memory storage devices; a graphical userinterface module stored in the one or more memory chips and executableby the one or more processors to generate data for displaying one ormore maps on the display, the one or more maps each comprising: aplurality of representations of geographic locations; one or moreindicators corresponding to one or more of the plurality ofrepresentations of geographic locations related to one or more qualityof user experience (QoE) metrics predicted for the computing device atthe geographic locations.
 2. The computing device of claim 1, whereinthe plurality of representations of geographic locations correspond toone or more of houses, businesses, and public use areas.
 3. Thecomputing device of claim 1, wherein a first indicator of the one ormore indicators comprises one or more network performance predictors forpredicting QoE metrics related to one or more network types.
 4. Thecomputing device of claim 3, wherein a first network performancepredictor of the one or more network performance predictors furthercomprises a first network issue predictor, the network issue predictorcomprising one or more of: a geographical issue indicator indicatingthat the a first network issue is related to geographical issues; adistance issue indicator indicating that the first network issue isrelated to a distance from a first geographical location to a source ofthe first network; and a location issue indicator indicating that thefirst network issue is related to the first geographical location. 5.The computing device of claim 3, wherein the one or more networkperformance predictors are related to one of: a 2G cellular network, a3G cellular network, a 4G LTE cellular network, a land-line basedtelephony network, the internet, and a wireless local area network(WLAN).
 6. The computing device of claim 1, wherein the one or more mapscomprise two or more overlaying maps; and wherein each map layer relatesto a different network type.
 7. A computing device comprising: adisplay; one or more processors; one or more memory storage devices; agraphical user interface module stored in the one or more memory chipsand executable by the one or more processors to generate data fordisplaying one or more screens on the display, the one or more screenseach comprising groups of icons, a first icon comprising one or more of:a first application icon related to a first application stored on theone or more memory chips and executable by the one or more processors;and a first application performance predictor, disposed in an overlyingmanner to the first application icon, indicating one or more predictedQoE metrics for the first application.
 8. The computing device of claim7, further comprising: a first application issue indicator disposed inan overlying manner to the first application performance predictor,indicating a source of an issue with the first application.
 9. Thecomputing device of claim 8, wherein the first application indicatorcomprises: a network indicator if the cellular network is the issue withthe first application; a provider indicator if the application provideris the issue with the first application; a user equipment (UE) indicatorif the UE is the issue with the first application; and an applicationindicator if the first application is the issue with the firstapplication.
 10. The computing device of claim 8, wherein the firstapplication performance indicator and the first application issueindicator are not displayed on the display if the first application isperforming nominally; and wherein the first application performanceindicator and the first application issue indicator are displayed on thedisplay if the first application is not performing nominally.
 11. Thecomputing device of claim 7, wherein the first application performanceindicator comprises: a green light if the first application isperforming nominally; and a red light of the first application is notperforming nominally.
 12. The computing device of claim 11, wherein thefirst application is not performing nominally if one or more of thefollowing occurs: the user equipment (UE) drops one or more calls in apredetermined time period from a cellular network; and the UE freezeswhile running the first application.
 13. The computing device of claim7, wherein the first application performance indicator comprises anapplication speedometer overlay.
 14. A system comprising: a processorfor collecting a plurality of data points and compiling a plurality ofdatabases; a network interface connecting the processor with any or allof one or more application providers, one or more user equipment (UE)applications, one or more internet service providers, or one or morecellular network providers; and one or more memory storage devices forstoring the plurality of databases, the plurality of databasescomprising: one or more application providers databases comprising aplurality of data points related to one or more UE applications; one ormore UE databases comprising a plurality of data points related tointernal and external performance data points for a plurality of UE; oneor more internet service provider (ISP) databases comprising a pluralityof data points related internet performance; one or more cellularprovider databases comprising a plurality of data points related tocellular network performance; one or more network metric databasescomprising a plurality of data points related to network quality ofservice (QoS) metrics; and one or more user input databases comprising aplurality of data points related to user quality of experience (QoE).15. The system of claim 14, wherein the plurality of databases compriseonly data points related to system faults.
 16. The system of claim 15,wherein the system faults comprise one or more of a dropped call, afailed call, an application freeze, or a network access issue.
 17. Thesystem of claim 14, wherein the data points related to internalperformance data points for a plurality of UE comprise one or more of:processor utilization rates, memory utilization rates, and bufferutilization rates.
 18. The system of claim 14, wherein the data pointsrelated to external performance data points for a plurality of UEcomprise one or more of: signal strength, network connects anddisconnects, dropped calls, and data throughput.
 19. The system of claim14, wherein the processor generates a QoE performance predictor for afirst UE based on the plurality of data points in one or more of theplurality of databases.
 20. The system of claim 19, wherein theprocessor provides the QoE performance predictor to the first UE via thenetwork interface; and if the QoE performance predictor predicts thatperformance will be less than nominal, a special service recommendation.